Meta Learning: Learning to Predict the Leave–one–out Error

We propose a meta learning framework, casting leave-one-out (LOO) error approximation into a classification problem. For Support Vector Machines this means that we need to learn a classification of whether or not a given Support Vector – if left out of the data set – would be misclassified. For this learning task, simple data set dependent features are proposed, inspired by bounds from learning theory and geometrical intuition. Our approach allows to predict the LOO error on unseen data with an astonishing degree of accuracy – as demonstrated in simulations. Comparisons to existing learning theoretical bounds, as e.g. the span bound, are given for model selection and LOO error prediction scenarios.